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PREFACE 



"Skill deterioration" has been an all-too-familiar problem for at least 
a generation of Naval officers who have participated in the management of 
the Navy's manpower and personnel. The examination of the problem docu- 
mented in this report was sponsored by the Systems Analysis Division of 
the Navy's Program Planning Office. The monitor of the effort was Charles 
R. Hoshaw of the Enlisted Plans Branch, Bureau of Naval Personnel. Both 
Hoshaw and CDR Fred J. Breaux of the Mobilization Planning and Programming 
Branch, Office of the Deputy Chief of Naval Operations (Manpower), were 
most helpful in providing the initial guidance and direction for this effort. 

Two thesis students of the authors, LT James E. Taylor and LT David 
M. Thalman, who participated in the project were most helpful in the litera- 
ture search and review, contacting current research activities, and providing 
the "inside view" of the problems associated with skill deterioration and its 
management in the Navy. Their thesis on the topic and participation in this 
study is another example of the close working relationship between the faculty 
and students at the Naval Postgraduate School on topics of current concern 
to the Navy and the defense posture of the United States. 

The addendum, which presents a formal model of the recommended study 
effort, was inserted at the suggestion of the study monitor after the initial 
review copy was circulated. It was prepared for a project briefing at the 
Naval Postgraduate School of RAMD C. A. Trost, the Director of the Systems 
Analysis Division (NOP-96), Office of the Chief of Naval Operations. 
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EXECUTIVE SUMMARY 



PURPOSE 

The purpose of this study was to recommend a feasible research and 
study program to develop the knowledge and methods necessary to manage 
the problem of skill deterioration among Navy enlisted personnel due to the 
nonutilization of their learned skills. 

PROBLEM 

As the technology of Naval warfare has increased in its complexity and 
rate of change, so has the difficulty in maintaining a sailor's skills and 
proficiency in the occupation for which he or she is trained. A major factor 
contributing to this difficulty is the nonutilization of learned skills for 
extended periods of time. This situation could develop as a result of an 
assignment outside of the person's specialty or because of a break in ser- 
vice. The situation is typical of sailor's in the Reserve forces whose skill 
utilization periods are much shorter than the periods of nonutilization. 
Actually, skill deterioration, itself, is a symptom. The decision maker's 
problem lies in its cure--training. That is, the decision maker is concerned 
with skill deterioration because retraining may be required to rehabilitate 
skills that have deteriorated due to nonuse and because a given amount and 
frequency of training must be provided the Reserves to prevent skill deteri- 
oration through excessively long periods of nonutilization. The problem is 
depicted in the accompanying figure. 

The elements of the problem are the current level of performance on the 
job, the skill deterioration curve, and the retraining curves at various 
periods of nonutilization. The level of performance on the job is the base- 
line from which the skill deterioration curve is derived and the criterion 
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Ideal (Hypothetical) 




Schematic Representation of the Skill Deterioration Problem. 



to be attained in retraining. However, determining this level is no easy 
matter in practice. As shown in the figure, an ideal or hypothetical level 
can be logically created, but the actual performance of job incumbents has 
been found to be around 25 percent below such ideal levels. The skill de- 
terioration curve is initially steep, but it should level off when the 
large decrement due to the immediate forgetting of poorly learned and 
practiced skills has passed. As the period of nonutilization continues, 
another factor enters the picture--a change in the job itself. This again 
leads to a rapid deterioration of skills due to their obsolescence. As the 
shape of the skill deterioration curve can be expected to reflect qualita- 
tively different sources of skill deterioration, so can the retraining 
curves be expected to differ according to the place on the skill deter- 
ioration curve when retraining commences. If retraining must overcome 
obsolescence, for example, training will consist essentially of new learning. 

There are many occupations in the Navy with quite specific skills 
being required in particular positions. These are reflected in the Navy 
Enlisted Occupation Standards and the Navy Enlisted Classification (NEC) 
system. The skill deterioration problem, in this context, is twofold. 

First, there is no way to determine what the skill deterioration curve may 
be for any one occupation and, consequently, the amount and type of re- 
training required. Second, the jobs and positions for which skill deter- 
ioration curves and rehabilitative training programs must be developed are 
very numerous. Thus, the skill deterioration problem has its qualitative 
and quantitative aspects. 

CURRENT STATE OF KNOWLEDGE 

The first phase of this study was directed at determining what was 
available in the scientific literature, in ongoing research activities, and 
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in the institutionalized data and information management systems of the Navy 
that could be used to define the skill deterioration problem and as guidance 
for its management. Unfortunately , existing, extensive reviews of the liter- 
ature covering a period of over 75 years could provide very little of direct 
application to the problem as stated. The emphasis in the literature is on 
skill acquisition, rather than retention; the basic research is more concerned 
with memory and forgetting processes rather than the results of the non- 
utilization of learned behaviors; and the experimental tasks that have been 
used have been simple and artificial. The literature on retraining is con- 
siderable, but it deals with the training in a new occupation of persons 
whose skills are no longer marketable, rather than the rehabilitative train- 
ing of persons who have lost their capabilities and skills. Scientists 
within the present research scene quickly acknowledge the problem of skill 
deterioration, but there is very little experimentation designed specific- 
ally to investigate it. Concern is especially high in the area of flying 
skills because of the reduction in flying hours to conserve energy sources, 
but the emphasis is on developing substitute (simulator) training, rather than 
on measuring the loss of skills. There are, however, a few interesting 
efforts that may provide some insight into the skill deterioration problem, 
but the specificity of the studies merely serves to accentuate the extremely 
long course that a strictly experimental program would take. 

The management information systems of the Navy were found to be organ- 
ized in a way that could not provide data to be applied to the problem of 
skill deterioration. These systems do involve the collection of task and 
performance data, but each process--such as NEOCS, NOTAPS, and PQS--serves 
a different master and together they do not provide a quantitative assess- 
ment of skill levels as they exist in the fleet for tasks essential to the 
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performance of prescribed missions. Thus, if baseline performance cannot 
be established, neither can the skill deterioration function or the require- 
ments for retraining. 

RESEARCH AND STUDY ALTERNATIVES 

These findings indicate the need for a very comprehensive program of 
data collection and research if answers to the skill deterioration problem 
are to be found. Three possible approaches were considered: a performance 

testing approach, a systematic job/task/skill analysis approach, and a 
snythetic, simulation-based approach. 

Performance Testing 

One deceptively simple and straightforward method recommended to us 
by many persons was a performance-testing approach. In essence, this method 
would test persons on the job and persons returning to the job to determine 
the skill loss due to nonutilization. The gap, if any, would be the retrain- 
ing that must be accomplished. The concept is excellent, but its implemen- 
tation is difficult. Performance testing that is criterion referenced 
embodies, in large part, a hands-on, job-sampling approach. Attempts to 
develop such tests for determining personnel readiness have met with much 
difficulty because the development and validation process necessarily in- 
trudes into the duty cycle of persons on the job. Even if such tests could 
be developed for the entire range of Navy occupations, the costs of estab- 
lishing, instrumenting, and operating testing facilities for evaluating 
the skill deterioration of persons who have been out of their occupational 
areas and are returning to them would be excessive. 

Job/Task/Skill Analysis 

The typical training approach to the problem of retraining lost or 
degraded skills would be to analyze jobs into tasks and subtasks and then 
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specify what skills and knowledge are required to perform the tasks. A 
criticality dimension may also be applied to the tasks to establish a 
priority of tasks to be trained. The argument put forth by this approach 
is that the skill deterioration and retraining problems cannot be diagnosed 
and remedied without a knowledge of the skills required to perform the tasks 
and their criticality. The problems associated with this approach are the 
difficulty in validating that the skills specified are those that are act- 
ually required--often a major experimental effort in itself, determining 
the level of skill necessary for satisfactory performance, determining the 
actual skill level of those on the job, and devising a method for measur- 
ing skill levels at a remote site in the case of those who have been out of 
the job and are returning to it. It is evident that this method involves all 
of the problems and costs of the performance testing method plus the time and 
costs in attempting to understand and control the underlying skill mechan- 
isms. However, the understanding achieved in one task or job could conceiv- 
ably be generalized to other tasks and jobs with a potential savings in 
effort. These methods would also provide a sounder base for prescribing 
retraining. 

Synthetic, Simulation-Based Approach 

In order to circumvent the costs and time involved in the preceding 
alternatives, a synthetic, simulation-based approach was considered. It 
is essentially a method for capturing the policy of those who make retrain- 
ing decisions now and who prescribe training programs for the various cate- 
gories of the Reserves. It systematizes what is being done by intuition and 
experience so that it can be applied by rule to any job or billet. Using 
personnel and training experts and experienced Navy supervisors, a list of 
job characteristics that would have a bearing on the skill deterioration 
and obsolescence rate for any and all would be created and statistically 



manipulated to result in a smaller list of orthogonal or independent job 
characteristics or clusters. These characteristics would be used to de- 
scribe synthetic jobs that have varying amounts of the characteristics. 

The experts would then rate the amount of skill deterioration that would 
occur for each of these synthetic jobs at various periods of nonutilization. 
Appropriate statistical methods would then be used to recapture the policy 
of the experts and supervisors so that the relative importance or weight 
of each of the "pure" job characteristics could be determined. The resulting 
vector of weights could then be applied to any job that could be scaled on 
the relevant characteristics to obtain its skill deterioration curve. Thus, 
the system would be efficient in that one set of weights could be applied 
to any job. 

The validity of the method could be determined by using it to make 
retraining decisions or training programs for the Reserves. Feedback from 
these efforts would be used to reevaluate and, if necessary, adjust the 
weights of the skill deterioration vector. This approach deals directly 
with the skill deterioration problem and will provide an operational tool 
within a relatively short period of time that has the benefit of being self- 
correcting. 

SUMMARY AND RECOMMENDATIONS 

Skill deterioration among Navy personnel due to the nonutilization 
of learned skills is a serious problem with no easy solution. The avail- 
able scientific literature does not provide much that can be used to solve 
the problem. Empirical approaches to determining skill loss for the entire 
range of Navy occupations would be an undertaking of considerable proportions 
that would have to be done on a continuing basis as jobs change. Without 
the long-range commitment of resources and continuing command support these 



methods will not produce the desired results. A short-range approach 
that attacks the problem directly using expert opinion seems to be the only 
feasible solution. Along with the implementation of the short-range 
approach, the Navy's performance measurement system (PQS) should be im- 
proved to permit a scoring of individual performance that reflects the 
actual capabilities of the individual. The NOTAP program should be im- 
proved so that tasks are consistently described across all occupations to 
permit the analysis and derivation of important, composite skills using 
appropriate statistical procedures. Eventually, then, the short-range 
approach using personal judgments can be replaced with actual data in sim- 
ilar categories from the job environment itself. In the meanwhile, de- 
cisions made on the basis of the interim method should be validated by 
feedback from the training or job environment and a sel f-correcti ng cycle 
be established. 
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INTRODUCTION 



This report summarizes work carried out in accord with our proposal 
entitled "Deterioration of Skills." The report reviews, in part, literature 
related to the proposed topic and discusses programs designed to provide 
better insight into skill deterioration and approaches which have the po- 
tential to aid in the interim and long-range to the retraining question in 
the Navy. 

Essentially, the task statement contained in the proposed study effort 
was as follows: 

(1) Establish a series of topics of interest which are of importance 
in skill deterioration. 

(2) Review the literature relevant to skill deterioration, job 
obsolescence, and nonutilization in the Navy's enlisted structure. 

(3) Define gaps and deficiences in areas of interest established 
above. 

(4) Suggest research efforts which will aid in closure of observed 
gaps and deficiencies. 

In the course of our activities we were exposed to a variety of views 
relevant to the topic of interest. These activities were the result of 
a literature review and discussions with individuals during travel. The 
discussions and activities of those visited are summarized in Appendix A. 
Later sections are devoted to a discussion of a proposed approach to the 
retraining problem and required study efforts. 

PROBLEM OF SKILL DETERIORATION IN THE NAVY 

In an organization as large as the Navy with its many diverse and 
unique operational activities coupled with the wide variety of operational 
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environments and complex of Naval systems, a premium is placed on the devel- 
opment and retention of a highly capable and skilled enlisted work force. 

One aspect of this total problem is the maintenance of an acceptable level 
of performance proficiency or skill in personnel necessary to fulfill war- 
time manpower requirements. The magnitude and significance of the problem 
becomes obvious when one considers that approximately 75 percent of the Navy 
occupational specialities requires the development and use of specially 
trained skills. The problem of availability of such personnel is compounded 
by the fact that at any given time approximately 25 percent of the skilled 
work force is assigned to responsibilities not involving skills and capabil- 
ities for which they were trained (Taylor and Thalman, 1977). Further, a 
significant component of the total work force in a wartime situation will 
consist of reservists, retired personnel, and prior service volunteers. 

As such, an emergency situation would find the Navy with a large element of 
its enlisted force experiencing various levels of skill deterioration and/or 
job obsolescence. 

Therefore, a significant problem for the modern technical Navy is one 
of assessing and analyzing the capability of naval personnel resources re- 
quired to maintain operational capability at an acceptable level. A major 
aspect of the problem involves specification of the nature and extent of 
retraining required to offset anticipated reduction in performance capacity 
resulting from nonutilization of skills and job obsolescence. 



2 



RELEVANCY OF EXISTING DATA 



As suggested earlier, our task was to involve, in part, a review 
of work related to the problem of skill retraining requirements following 
periods of nonutilization and/or job obsolescence. In terms of applicabil 
ity to the stated objective, the literature review was disappointing. 
Current and past efforts devoted to the study of skill degradation have 
been rather fragmented and frequently simplistic. There does not appear 
to have been an organized research program designed for the purpose of 
providing research data necessary to establish and implement an effective 
diagnostic and implementation program in skill maintenance. In general, 
the present state of the art consists of appreciation of the complexities 
involved in performance rather than an understanding of the variables and 
their interaction, particularly as these variables relate to maintenance 
of performance proficiency. Fleishman, et^ al_. (1973), for example, have 
concluded that it is virtually impossible to translate, let alone apply, 
many of the terms and concepts developed in learning laboratories to train 
ing programs applicable to real world situations. Too often, reports 
survey available literature and conclude with a categorization of those 
parameters considered to be of importance. Infrequently is there a 
discussion of possible interrelationships between parameters as they would 
apply in the real world. 
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LACK OF A SYSTEMATIC MILITARY PROGRAM 



Paralleling the absence of research programs directed at the skill 
deterioration and retraining problems is the lack of an organized, systematic 
effort to obtain empirical evidence from ongoing Navy programs and activities 
to solve problems in these areas. The 3M programs, for example, are designed 
to provide data to improve the maintenance and readiness of Navy material, 
and aviation safety programs exist to provide information to improve the 
safety and performance of aircraft, flight personnel, and their associated 
systems. Such organized efforts do not exist among the manpower and per- 
sonnel programs to determine how much training is required to maintain and/or 
rehabilitate requisite skills when they are not habitually used in a person's 
daily work activities. 

At the present time, a program exists for rehabilitating other service 
veterans (OSVETS) who reenlist in the Navy after a period of broken service. 
There are two sites, San Diego and Great Lakes, where those with less than 
two years of broken service are given training in general military subjects 
peculiar to the Navy--such as fire-fighting, damage control, shipboard safety, 
and Navy human resources management. Those who have greater than two years 
of broken service are sent to a Recruit Training Center to start at the be- 
ginning with new recruits. In either case, the individual usually proceeds 
through an initial-training, specialty school following the first phase 
of rehabilitation training. Navy personnel on active duty returning to their 
specialties after a period of nonutilization and Navy reenlistees with 
broken service are usually sent through a rehabilitation program, often 
integrally located within the receiving command, prior to assignment to 
regular duty. These programs are designed to handle the retraining re- 
quirements of once- trained personnel entering or reentering a skilled 
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occupation in the active Navy, but there is not the necessary organization 
and planning of these activities to provide the information necessary for 
their improvement and for application of the findings to the Reserve com- 
ponent training of the Navy. However, personnel experienced in the opera- 
tion of these programs should be able to provide qualitative insight into 
the parameters that seem to affect skill deterioration and retraining. 

There is another area where several programs exist that are of direct 
relevance to the skill deterioration problem. The area is concerned with 
specifying and describing the occupations and jobs that make up the enlisted 
structure of the Navy. As in civilian industry and commerce, there are 
several approaches depending on the purpose and objectives of the various 
efforts. There is, first, the Navy Occupational Task Analysis Program 
(NOTAP) that is a computer-based, information system. It generates, main- 
tains, and analyzes tasks performed within Navy occupations. The descrip- 
tion is detailed and behavioral, and the prime user is the training community. 
Then there is the Navy Enlisted Occupational Classification System (NEOCS) 
that is stated in terms of the functions performed and the knowledge re- 
quired for a particular occupation. The prime user is the personnel manage- 
ment community as it deals with the selection, classification, and assignment 
of individuals. There is also the Performance Qualification Standards (PQS) 
which are a combination of textbook-type knowledge requirements and a 
hands-on demonstration of proficiency by a job incumbent. The orientation 
is toward individual career development and advancement and unit readiness. 
Finally, there are the squadron, ship, and shore manning document programs 
that are based on job sampling methods and the 3M maintenance requirements 
to generate manpower requi rements . 

All of these programs have the common characteristic of defining and 
setting standards for Navy jobs and occupations, and yet none or any 
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combination of them will define precisely what degree of skill or what 
combination of capabilities are actually necessary to perform a job ade- 
quately in the Navy. Stated in another way, they cannot provide precise 
information as to how well incumbents actually perform their jobs. Unless 
there is a precise way for determining performance requirements that can be 
stated quantitatively, it is not possible to determine when and to what 
degree skill has deteriorated. Neither is it possible to determine when 
rehabilitative training has served its purpose. Perhaps the lack of inte- 
gration of these many activities is a reflection of the excessive fragmen- 
tation, in the past, of the Navy's manpower requirements, training, and 
personnel management functions, which is currently being corrected 
(Salzer, 1976). 

RESEARCH NEEDS 

Emphasis in the Navy has consisted of the development of programs 
designed to impart skills and knowledge to the recruit which will enable 
him to reach a level of performance competency sufficient to maintain fleet 
readiness. The current problem consists of proficiency degradation as a 
result of nonutilization of learned skills and/or technological obsoles- 
cence. Therefore, the problem is one of maintenance or updating of a pre- 
viously learned skill as opposed to initial skill acquisition or develop- 
ment. 

Goldstein (1974) has suggested that there are three basic components 
in any attempt to assess retraining requirements--organization, job/task, 
and the recipient. In terms of the Navy, these components can be trans- 
lated into (1) operational objectives of the Navy, (2) analysis of job/ 
task/skill, and (3) analysis of the individual sailor. 

Analysis of Navy needs and objectives is essential in the develop- 
ment of measures of effectiveness for eventual evaluation of a retraining 
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program. Organizational analysis will provide criteria by which the success 
or adequacy of any proficiency maintenance program can be judged and will 
ensure an understanding on the part of all units that can be expected to 
interact with the retraining program (Goldstein, 1974). 

Therefore, from an organizational point of view, it is possible for 
the Navy to hypothesize a potential level of proficiency which is ideal 
and represents 100 percent effectiveness. However, in most man-machine 
systems the achievement of maximum proficiency is rarely observed. The 
influence of within and between subject variability, environmental, situa- 
tional, social, and organizmic stressors all tend to degrade potential pro- 
ficiency to a level somewhat less than the hypothetical ideal. Hopefully, 
however, observed proficiency can be maintained at a level necessary for 
acceptable operational readiness. Figure 1 presents a graphical repre- 
sentation of the maintenance problem. 

Organizational analysis is required to ensure that expectations on 
the part of the Navy are not unreasonable. In the absence of realistic 
proficiency requirements a potential conflict exists between objectives 
of the Navy and a retraining effort designed to maintain proficiency. 

The job/task/skills analysis of Navy need assessment will provide 
information necessary to fully understand the requirements for satisfactory 
performance in the maintenance of operational readiness. This aspect of 
assessing retraining requirements would involve a series of steps designed 
to define and describe jobs/tasks in terms of criticality, frequency, and 
susceptibility to obsolescence. As such, a distinction must be made be- 
tween job analysis, task analysis, and skill analysis. The Navy must 
develop a program which will identify the what ( i . e . , job and task) and 
allow for determination of how an operator achieves satisfactory performance 
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Ideal (Hypothetical) 
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Figure 1. Schematic Representation of the Skill Deterioration Problem. 



(ski 1 1 ) (Sal vendy & Seymour, 1973). The problem of maintaining proficiency 
becomes one of integrating the "what and how" under different times, places 
and jurisdictions (Silvern, 1970). 

The third component in assessing retraining requirements is the most 
difficult and most formidable. The Navy must develop a program that will 
examine performance standards and capabilities of the target population 
for use in the assessment of retraining/proficiency maintenance require- 
ments. The primary objective of this portion of the analysis is to deter- 
mine behavioral characteristics which have been learned and retained by 
individual performers (Goldstein, 1974). The objective of a retraining 
program is to return individual performers who are now or have been oper- 
ators in the system to full duty. This portion of the need assessment 
analysis will provide data essential for the design of a retraining effort 
which considers behavioral characteristics and proficiency levels of groups 
and individuals and provides the basis for tailoring retraining require- 
ments to the individual. 

The systematic research program suggested above requires a major com- 
mitment of resources that is sustained over a long period of time. Research 
and development in the Navy, as in the other services, tends to be spasmodic 
with the recurring sacri fi ce--temporari ly or permanently--of long-term 
programs to short-term needs. A long-term skill deterioration research 
program could well meet such an eventuality. One underlying cause for these 
disruptions is that decision makers must provide immediate answers, if not 
solutions, to current problems. Thus, in the fact of real-world demands on 
decision makers, it would be appropriate to consider the possibility of 
short-term research that is designed, primarily, to help decision-making 
and planning with respect to the skill deterioration and obsolescence issue 
without, necessarily, providing any great understanding of the problem. 
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This study, therefore, also examines what could be done in the area of 
research into the skill deterioration problem with immediacy as the mo- 
tivation, rather than understanding. 
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SKILL DETERIORATION LITERATURE 



SKILL RETRAINING 

A number of problems present themselves upon examination of studies 
dealing with the general subject of skill. As suggested above, data pur- 
ported to deal with the question frequently does not have a clear-cut 
application to the "real-world." Data collected in sterile laboratory 
settings are frequently too simplistic and lack application to problems in 
the fleet. Further, the literature tends to use the terms skill, pro- 
ficiency, aptitude, task, job ability and performance interchangeably. 

This situation results in considerable confusion and prevents the develop- 
ment of an organized set of research data on the subject of skill specif- 
ically. Fleishman, et^ al_. (1973) have defined this problem earlier; how- 
ever, the present effort would suggest that the problem remains and repre- 
sents an area in which a solution is required if meaningful data applic- 
able to the operational Navy is to be forthcoming. 

Considerable effort has been devoted to the study of acquisition of 
skill. One of the most notable collection of papers on this subject is 
Bilodeau (1965). However, as the purpose of the current effort was on 
the retention or maintenance of skill, a comprehensive review of skill 
learning research will not be included here. 

Perhaps the first major effort was a review by Naylor and Briggs 
(1961) on the retention of flight skills conducted for the Air Force. 
Naylor and Briggs categorized variables of interest into task, learning, 
retention interval, and recall. While their effort was comprehensive, the 
emphasis was on laboratory research as opposed to operational application. 
This emphasis combined with the report's verbal learning and memory orien- 
tation limits its utility to the present effort. 
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A second major effort involved the investigation of skill loss in 
Navy personnel when assigned to non-rating related billets (Rose & Turner, 
1967). This study represents an attempt to examine the retention or main- 
tenance of technical skills in the operational Navy. One important aspect 
of the report was the authors' differentiation between "skill loss" and 
obsolescence. They suggest that a distinction be made between the inabil- 
ity of Navy personnel to perform on a job through lack of practice (non- 
utilization) and inability resulting from technological changes (obsoles- 
cence). 

As a result of their review, Rose and Turner concluded: 

(1) Discrete tasks involving sequential steps incurred consider- 
able loss of skill over time, while continuous tasks suffered 
little decrease in retention. 

(2) Job skill research has been predominantly oriented toward 
operational skills and not maintenance skills. 

(3) Few reports exist on skill loss due to nonpractice of those 
ski 11s. 

Gardlin and Sitterley (1972) conducted a review of skill retention 
literature and concluded that retention variables should be separated into 
four major categories: (1) amount of training, (2) duration of retention 

interval, (3) task organization, and (4) task environment. 

Gardlin and Sitterley suggest that skill retention would appear to 
be functionally proportional to amount of original training, extent of task 
organization, and degree of similarity between simulation and operational 
setting. Further, skill retention is most likely inversely proportional 
to duration of the nonutilization period. The primary factor in the pre- 
diction of skill retention, according to Gardlin and Sitterley, is the 
final level of skill acquisition prior to nonutilization. 
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Gardin and Sitterley make an important point when they suggest that 
training/retraining programs should be based on previously identified 
critical performance dimensions. Silvern (1970) made a similar suggestion 
in a discussion of maintaining trained proficiency. Therefore, essential 
task elements which determine success or failure of a mission must be 
identified and stressed in proficiency analysis, training and/or retraining. 
Performance measures should be designed to analyze an operator's capacity 
to perform those aspects identified as being critical. 

Prophet (1976) has conducted a comprehensive literature review of 
long-term retention of flying skills. The Prophet effort was oriented 
around the following categories of subject matter: (1) general retention 

factors, (2) task or skill factors, and (3) retraining factors. 

In agreement with Gardlin and Sitterley (1972), Prophet has suggested 
the best predictor of performance following nonutilization of skills is the 
highest level of skill acquired prior to that interval. However, no 
precise statement can be made relative to the duration of retention. At 
best, one can conclude from available data that the longer the period of 
nonutilization the greater the decrement in proficiency. However, the 
question of proactive and retroactive inhibition on retention has not been 
examined sufficiently to allow for the drawing of conclusions as they relate 
to technical skills. 

RETRAINING AND OBSOLESCENCE 

The majority of work completed on the subject of retraining is not 
generally applicable to the present task. The primary emphasis in what has 
been called retraining research has been directed toward preparing an 
individual for entry into a new job specialty subsequent to job displacement. 
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As such, research has concentrated on development of skills and knowledge 
in what is essentially a new worker rather than updating or maintaining 
proficiency of that worker for the existing job function. Therefore, 
"retraining" research might better be described as "training" where the 
primary interest is in development of new skills and imparting new knowledge. 

However, the subjects of proficiency maintenance, technical obso- 
lescence, and nonutilization have been recognized as crucial problems in 
the area of personnel training and development in industry (Landy & Trumbo, 
1976; Dubin, 1972; Siefert, 1964; Dubin, Shelton, & McConnell, 1974). 

These efforts have shown that performance/proficiency degradation can result 
from (a) rapid changes in technology and knowledge making current skills 
insufficient, and/or (b) loss of effectiveness can result from nonuse. 

Both forms of proficiency decrement exist in the modern Navy and can be 
defined as obsolescence resulting from accelerated technical change and 
nonutilization of learned skills. 

Zelikoff (1969) has suggested that a major need in the area of ob- 
solescence and retraining is the development of a method of quantifying the 
problem. Mali (1969) has proposed an obsolescence index (01) for engineers: 

_ current knowledge understood by engineers 
current knowledge in the field 

However, employment of such an index requires assessment techniques capable 
of specifying current knowledge. Solution of this problem will require 
the development of research efforts designed to identify and classify the 
nature and cause of obsolescence in Navy job specialities and situations, 
e.g. reservists, personnel temporarily assigned outside their billets, etc. 

SUMMARY 

In summary, existing literature concerned with proficiency mainten- 
ance rarely exists in a form readily usable by the Navy. The literature 
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fails to provide what Glaser (1960) has considered required information for 
performance measurement and proficiency maintenance. Applied to the Navy's 
retraining effort, literature is deficient in the following fundamental cate- 
gories (Glaser, 1960): 

(1 ) Definition of performance to be evaluated 

As a result of the fragmentation of research data, available information 
does not allow for the integration required for measurement and analysis of 
performance characteristics of interest to the Navy in a proficiency main- 
tenance effort. On the basis of existing literature it is not possible to 
specify levels of proficiency. Rather, the concentration has been on measure- 
ment prior to and following training and as such is frequently not applicable 
to the current problem. 

One major problem in performance evaluation is the absence of ac- 
ceptable standards in many Navy job specialities. Industry frequently 
employs an "experienced worker standard" (EWS) in personnel evaluation. 

It is necessary that performance standards and methods of measurement 
be developed which are applicable to specifying retraining requirements 
following periods of nonutilization or technological obsolescence. Avail- 
able literature does not provide a basis for developing such an "EWS" 
for the Navy. 

(2) Determination of performance aspects to measure 

Performance proficiency can be measured in terms of final product or 

behaviors involved in performance proficiency. The literature does not 
provide a basis for determining those aspects of performance which are most 
relevant and therefore enable discrimination between effective and ineffec- 
tive workers. Information is lacking which will allow for specifying those 
aspects which are representative and critical and therefore worthy of 
measurement and evaluation. That is, in some job specialities the important 
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aspect may be the end product, while in other situations or jobs, ongoing 
behaviors may represent the critical aspect of proficiency. 

(3) Measurement methods 

Existing literature does not specify techniques required to measure 
observable behaviors and/or end products which need to be identified, re- 
corded, and measured for subsequent ordering along an evaluative scale. 
Information is needed which will specify performance features to be evalu- 
ated and specify standards against which it will be compared. 

(4) Reliability of measurement 

Existing 1 iterature with its concentration on "skill acquisition" 
has not provided a basis for measuring proficiency degradation resulting 
from nonutilization and/or job obsolescence. Measures must be examined in 
terms of their ability to measure and assess degradation following non- 
utilization or resulting from obsolescence, and not the ability of a train 
ing program to impart new knowledge to the uninitiated. 

(5) Validity of measurement 

The literature would suggest that research has concentrated on devel 
oping measures which are frequently ends in themselves. As suggested in 
category (1), development of standards and criteria in terms of system 
performance in the "real world" as opposed to evaluation of training or 
retraining programs is required if measurements are going to be valid in- 
dicants of system efficiency. Measures based on evaluation of a training 
program are of little utility in assessing retraining problems as they 
apply to the operational readiness of the fleet. 
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CURRENT RESEARCH EFFORTS 



OVERVIEW 

Given the status of the extant scientific literature, described above, 
an effort was made to determine whether any current research programs might 
be applicable to the skill deterioration problem. The search was limited 
to Defense activities with the expectation that ongoing research in the 
civilian sector would not differ significantly from the findings of the 
literature survey. Program summaries of research and development activities 
of the services provided one source of information. Personal contact with 
research personnel in the laboratories and study organizations was another 
important source. Finally, the two graduate students working on this pro- 
ject with the authors were sent on a site visit to the many contacts that 
were established. Their record of these visits is presented in Appendix A 
to this report. 

In general, ongoing research is oriented towards developing training 
techniques, equipment, and training systems. The emphasis continues to be 
on the acquisition of skill, as this is the training mission, rather than on 
its retention. 

ARMY FIELD STUDY OF ANTIAIRCRAFT MISSILEMEN 

There is, however one current research effort that deals specifically 
with skill loss and the role of refresher training in the maintenance of 
skill. A large field experiment has been designed and is already in the 
data collection stage at the time of this writing. The program was 
initiated because the Army has found that trainees forget much between the 
time they finish a school, take leave, and then report to their unit 
assignments. The project is in the individual training and performance 
evaluation area of the research effort of the Army Research Institute (ARI) 
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for the Behavioral and Social Sciences. Dr. Milton Katz is the overall 
program director for individual training and performance evaluation, and 
Dr. Joyce Shields is the project leader. 

The ARI study has an ingenious design utilizing soldiers trained in 
the 16P and 16R military occupational specialty (MOS) fields. Soldiers 
with one designation fire the Chaparral missile, and the other, the Red-Eye 
missile. Both MOS are trained in the same course at the Army's Antiaircraft 
Artillery School at Ft. Bliss, Texas. Accordingly, this permits a direct 
comparison within the same individual of skill loss due to nonutilization 
of learned skills, proficiency level in the assigned duty vs. proficiency 
at the completion of training, and the retraining necessary to achieve a 
required proficiency level as evidenced by the performance on the missile 
that he fires on duty. The nonutilization periods are relatively short, 
being 0, 1, 2, and 4 months after completion of training because the em- 
phasis is on refresher training and the maintenance of skills, rather than 
on the problem of extended periods of nonutilization. The findings of the 
study will, however have direct relevance to the question of skill deteri- 
oration and retraining, especially as it may be generalized to skill main- 
tenance in the Reserve. 

One aspect of this ARI study has been considerably facilitated be- 
cause the individual training research program includes a major effort on 
the study, analysis, and creation of criterion-referenced tests in training. 
Criterion referenced means that the trainee is tested by performing essen- 
tial, job-related tasks, rather than being normatively scored on a paper- 
and-pencil test of knowledge about the subject. But here again, it should 
be noted that to test a trainee on his ability to perform as a missileman 
at the conclusion of training is not the same as testing his ability to do 
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the job, since he will be required to fire only one of the missiles. 

While a great deal has been done on creating and applying criterion- 
referenced tests, these tests serve, primarily, as terminal objectives for 
training, rather than a definition of required job performance. 

PERFORMANCE TESTING RESEARCH 

An area closely related to criterion-referenced testing in the train- 
ing environment is the new interest in performance measurement and evalua- 
tion in the operational environment. Several behavioral scientists engaged 
originally in developing criterion-referenced testing were used by the Army 
to develop its Skill Qualification Test (SQT), which plays essentially the 
same role as the PQS in the Navy. The SQT program, which is supported 
through the Army's Training Support Center--as the PQS is similarly supported 
by Navy Training Support detachments— may have batteries of tests that are 
more psychometrically sophisticated than the PQS. The Navy, however, has 
a performance evaluation, research program at the Navy Personnel Research and 
Development Center (NPRDC) that is designed to provide a more precise 
evaluation of the performance capabilities of duty sailors than the PQS (Laabs, 
Main, Abrams, & Steinman, 1975) in order to create and prescribe the type 
of training that will lead to increased personnel readiness. The program has 
demonstrated how difficult it is to develop a prototype test for a single 
rating that truly reflects the incumbent's job demands (Laabs and Panel 1, 

1976). It has also demonstrated the need for adequate equipment that is 
representative of the duty environment if a true hands-on evaluation is 
to be conducted at a testing site other than the duty station itself. The 
NPRDC program has also indicated the large number of sailor-hours that 
would be required to develop and validate a test battery for a particular 
rating, and by inference, the large number of sailor-hours that would be 
required in an operational performance-testing program. In fact, on one of 
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its field studies, the NPRDC research team was not permitted to use duty 
personnel for performance testing owing to the encroachment on duty time 
(Laabs, Harris, and Pickering, 1977). The research team had to settle for a 
penci 1-and-paper test with much smaller demands on duty hours and organiza- 
tional equipment. The implications of this research and development program 
are that a positive command decision would be required at a very high level 
if evaluative, performance-testing programs are desired for determining the 
on-the-job proficiency of the Navy enlisted force because the costs are very 
high. And yet, an obvious and simple answer to the skill deterioration problem 
that is often proposed is to test the individual to determine whether he or 
she could be sent directly to a duty assignment or whether a period of re- 
habilitative training is necessary. 

The performance testing research at NPRDC has also shown that duty 
personnel cannot perform once-learned tasks if they are not required to per- 
form them on the job (Laabs, Pannell , and Pickering, 1977). For example, cer- 
tain technicians on nuclear submarines learn to test and maintain missile 
test and readiness equipment. Some of the equipment is so reliable, however, 
that they are never required to troubleshoot and repair the equipment. Tested 
on the procedures required to do so, they inevitably fail. A self-instructional, 
remedial study guide, on the other hand, that was designed to be used during 
a cruise was sufficient to enable those who had used it to perform the repair 
functions adequately upon subsequent testing. These findings have very sig- 
nificant relevance to this study. First, as mentioned early in this report, 
there is an ideal and an actual performance level for duty billets. The 
former may be used for developing training documents, but the latter is a 
more realistic baseline for determining skill deterioration and a retraining 
criterion. There is, apparently, considerable distance between the two 
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curves depicted in Figure 1. Of 305 BTs who were given a diagnostic test 
on job related behaviors, only 10 percent passed all modules of the test 
at a minimally acceptable cutoff score. Of those who did not pass all 
14 modules, the score on the basic mechanical procedures test of the modules 
was only 70 to 75 percent correct. These findings, if replicated in other 
situations, would suggest that the actual performance capability of duty 
sailors is only about 75 percent of the tasks that may be included in, say, 
the NOTAP description of the job. The second significant point to be made 
from the results of the NPRDC studies is that once-learned skills of a 
procedural nature can be rapidly reactivated with appropriate job-performance 
aids or self-study materials. Accordingly, the retraining requirement is 
also dependent on factors other than skill deterioration, such as the avail- 
ability of documentation and job-performance aids at the duty site. 

TWO AIR FORCE PROGRAMS 

There is concern in all of the services regarding the maintenance of 
flying skills with the serious reduction in flying hours as a result of 
the high cost and scarcity of fuel. The general approach to the problem is 
towardever more sophisti cated simulators for individual and team training. 
Simulators are also expensive in terms of procurement and operation, but 
there is limited, although generally positive, data on their ability to main- 
tain flying skills, particularly of a procedural nature. What is needed is 
a better understanding of the more critical component skills involved in 
flying and their perishability and retrainabi 1 i ty . The USAF Human Resources 
Laboratory (HRL) on flight training at Williams AFB, Phoenix, Arizona, is 
attempting to investigate these areas with a special interest in skill- 
maintenance training (Eddowes, 1977). While there are no enlisted skills in 
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the Navy that are strictly comparable with those among Air Force pilots, 
general principles with reasonable generality are expected to appear as a 
result of a systematic investigation of flying skills. 

At the USAF URL at Brooks AFB, San Antonio, Texas, where research is 
conducted in the area of Air Force personnel, a research program is being 
conceived to investigate systematically the perishability of skills as a 
function of specific parameters (Christal, 1977). For example, there may be 
a relationship between the shape of the learning (skill acquisition) curve 
and the retention (perishability) curve. Since the rate of learning and the 
terminal level of skill acquisition is closely related to aptitude, so must 
aptitude have a strong influence on the retention of learned skills. The 
potential benefit of research following this approach is that broadly applic- 
able functions could be discovered that would be of considerable utility in 
managing the skill deterioration and retraining problem areas. At the time 
of this writing, data were being generated for the acquisition curve at the 
USAF HRL, Lowry Field, Colorado (Ward, 1977). 



SUMMARY 

In summary, many training activities express a concern with the reten- 
tion of training, but research and development continues to emphasize the 
acquisition of skills. There are a few, individual projects at the per- 
sonnel research laboratories of the Army, Navy, and Air Force that deal 
directly with skill deterioration and retraining. In the Navy, the per- 
formance testing research at NPRDC has provided insight into the difficult 
problems associated with a testing approach, shown that personnel who are 
apparently performing acceptably in their jobs do not, and cannot, perform 
many of the skills they were taught and tested on in training, and have 
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indicated that once-learned, nonutil ized, and forgotten skills can be 
effectively rehabilitated with self-study materials. 
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I 



REQUIRED STUDY EFFORT 



REQUIREMENT FOR A LONG-RANGE PROGRAM 

The preceding sections have shown the scope and magnitude of the prob- 
lem the Navy faces in attempting to manage the problem of skill deteriora- 
tion. Essentially, the question is as broad as "How much and at what 
intervals is training necessary to maintain the requisite skills for a 
ready Navy?" Unfortunately the research on skill retention, training, and 
performance measurement was found to be quite fragmented and parochial and 
only relevant to specific problems and programs at very low levels of 
aggregation. At the highest levels of aggregation, an occupation could 
be the object of attention; at lower and more prevalent levels, the object 
of study would more likely be the acquisition and/or retention of skill in 
a particular task. Similarly, ongoing occupational programs in the Navy's 
manpower and personnel areas are fragmented and are designed to serve the 
immediate needs of the sponsoring activity. Thus, there is no link between 
the Navy's NOTAP program and the PQS program when, in actuality, they both 
deal with the activities and functions of a sailor in a particular occupa- 
tional category. In order to make inroads into reducing the skill deteriora- 
tion problem to a manageable state, a long-range orientation that attempts 
to direct and integrate research, plans, and programs towards its solution 
is necessary. A long-range program could, however, include short-range 
goals that attempt to gain better insight into the skill deterioration 
problem and, hopefully, to find interim solutions that can be used in the 
routine management and operation of the Navy's manpower and personnel 
systems . 
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ELEMENTS OF THE LONG-RANGE PROGRAM 



The study program should attempt to make use of the many ongoing 
activities that could contribute to an understanding and management of skill 
deterioration with respect to the Navy's entire enlisted structure. Eventu- 
ally, the various parts of the operating programs should be incorporated 
into an aggregated model of the manpower/personnel system, such as the 
NAMPS program. But in order to achieve this end, the long-range program 
should have some hierarchical goals in a time dimension. Called phases, the 
time-oriented goals might take on the following form. 

Phase I. As stated above, an immediate, ad hoc, interim solution is 
desired to provide a framework or model for the overall program, to facili- 
tate understanding and progress, and, at the same time, to provide guidance 
for the decision maker who must manage and operate the force on a continu- 
ing basis. The only way that this can be done is to use the judgments of 
experienced professional and military personnel as the basic data from which 
to build this short-term solution. A specific aim of the short-term effort 
should be the development of a method for estimating skill deterioration as 
a function of the duration of nonutilization and the original skill level of 
the individual. The objective of this study is to make specific recommenda- 
tions for such an effort. When the derivation of the skill deterioration 
curve has been accomplished, the same general methodology could be used to 
estimate the amount and kind of retraining that would be necessary to re- 
establish the original skill level at any point in the skill deterioration 
curve. At an early point in the skill deterioration curve, the problem could 
be restated as the amount and type of training necessary to maintain the 
desired level of skill. 
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Phase II. The objective of Phase II would be to develop and institu- 



tionalize a system for measuring and scoring performance that could be used 
to set a standard for required job performance. The PQS exists and should 
be used as the starting point. The PQS must be scored on a Guttman scale 
(Edwards, 1957) in which the performance items are sequenced according to 
their importance or necessity for adequate job performance. In such a 
scale, a particular score can be obtained in only one way and represents an 
individual's point on the scale. One point on the scale could represent 
an entry level of proficiency, another a minimally adequate level of profi- 
ciency, and another an acceptable level of proficiency. Without a quanti- 
fied standard, there would be no basis for estimating deterioration. The 
administration and scoring of the PQS should be better standardized to in- 
crease its reliability and, particularly, its validity with respect to per- 
formance that is actually required on the job (Laabs, Panell , and Pickering, 
1977). The many recent studies on criterion-referenced measurement should 
be consulted in this regard. 

Phase III. The objective of this phase would be to upgrade the 
task analysis and occupational analysis systems of the Navy so that more 
sophisticated analyses are possible, such as those required to make infer- 
ences across occupations. The NOTAP system should incorporate a method for 
determining the relative importance of the tasks in the inventory, and the 
method should be further refined so that occupations can be clustered accord- 
ing to their similarities and important job dimensions can be determined 
across many or all occupational fields. At the present time, clustering 
programs are appropriate only within an occupation. The reason for estab- 
lishing these capabilities is to provide a framework for determining skill 
deterioration functions using the PQS scores as the criterion measure. (The 
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concept of determining skill deterioration functions in this manner is 
developed below in the section dealing with the short-range program.) 

Phase III can be pursued concurrently with Phase II, and there should be 
liaison between the two efforts to establish a consistent framework. For 
example, the scaling of the PQS items should be consistent with the relative 
importance of tasks determined by the NOTAPS method. 

Phase IV. The purpose of Phase IV would be to gather empirical data 
through the PQS system in such a manner that the skill deterioration functions 
can be updated and refined on the basis of experience. Depending on how 
individuals are handled on their return to a specialty area after a period 
of nonuse, empirical evidence could be generated as to the efficacy and 
necessity of various retraining decisions. An adequate management informa- 
tion system would be required to integrate, develop, and store this infor- 
mation. Since this phase is far removed from the limited objectives of 
this study, it will not be further explored at this time. 

OUTLINE FOR A SHORT-RANGE STUDY EFFORT 

The purpose of the short-range study effort is to develop a schema 
for determining a skill deterioration function that could be applied to the 
entire array of skills in the Navy's enlisted classification system. This 
section will detail the steps that could be taken, using expert judgment, 
to arrive at such a skill deterioration function. The skill deterioration 
function, once obtained, could be used as a basis for determining retraining 
requirements using similar methods of eliciting and processing expert judg- 
ment. The steps in the process are outlined in Figure 2 and described 
below. 
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Determine what job character- Group interviews with pan- List of characteristics 

istics affect skill deterio- els of expert judges in that affect skill deteri- 

ration. training, manpower, and oration rate. 
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Model Development 



Step 1 . Convene professional and military personnel who can be consid- 
ered experts in the areas of training, manpower, and personnel to develop a 
candidate list of characteristics that affect the degree to which a particu- 
lar job would be subject to skill deterioration with nonutilization of the 
acquired skills. Various techniques could be used to elicit such character- 
istics. For example, critical incidents might be obtained and retranslated 
into a list of characteristics . The requirement for a quick reaction or turn- 
around time might be one basis on which jobs could be differentiated for 
skill deterioration potential. The presence of job performance aids, check- 
out lists, and manuals would also affect skill deterioration by negating small 
amounts of forgetting. Hopefully, a list of some 20 or 30 characteristics 
would be obtained. 

Step 2. Step 1 was designed to be overly inclusive to prevent the 
loss of leaving out potentially important characteristics. The task is 
now to reduce the redundancy and create maximally independent (orthogonal) 
characteristics that would be much fewer in number. The first step in 
accomplishing this objective is to obtain a proximity matrix for the many 
characteristics. This could be done by having the panel scale the related- 
ness (say, from -1 to 1) between any pair of characteristics. 

Step 3. Multidimensional scaling or factor analytic procedures could 
be used to reduce the proximity matrix to clusters or groups of characteris- 
tics that have much in common. If the groupings are complex, the judges 
could be asked to help identify and interpret them. The product will be a 
set of relatively orthogonal characteristics that will be called skill deter- 
ioration factors for convenience. 
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Step 4. This step begins the process of obtaining the "policy" that 



expert judges would use in evaluating jobs for their skill deterioration 
potential on the basis of the obtained set of skill deterioration factors. 

To do this, a large number of hypothetical jobs would be created by randomly 
assigning scaled values (say, from 1 to 10) to each of the skill deteriora- 
tion factors. The overall methodology is explained in a paper by Christal 
(1967), and the use of hypothetical cases is similar to the hypothetical 
ships created by Borman and Dunnette (1974) to determine the personnel 
readiness status of ships. 

Step 5. The hypothetical jobs are given to a panel of expert 
judges whose only task is to evaluate each job for the skill deterioration 
that would occur with nonutilization of learned skills. They will be asked, 
however, to make judgments based on whether the incumbent was in paygrade E4 
or lower or in paygrades E5, E6, and E7. These correspond to the apprentice 
and journeyman/supervisor levels recommended by the Navy Enlisted Occupa- 
tional Classification System (NEOCS) Study Group (1974). In accordance with 
the recommendations of the NEOCS Study Group, the enlisted supergrades, 

E8 and E9, will be considered as managers with competence in managing 
broad occupational areas. Because of their overlap with many specific 
job areas and their relatively small absolute numbers, they will not be 
considered in the recommended study effort. Additionally, within each 
of these classifications, hereafter referred to as experience levels, the 
judges will be asked to determine the amount of expected skill deterioration 
at periods of 1 , 2, 3, 4, and 5 years of complete nonutilization of job 
skills. Thus, each judge would evaluate each hypothetical job by two 
experience levels and 5 yearly periods. The degradation could be in terms 
of percentage loss in skills from an original level of 100 percent. 
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Step 6. Using multiple regression and a least squares criterion, the 
judges' evaluations of the hypothetical jobs in Step 5 will be regressed on 
the skill deterioration factors generated in Step 4 for the jobs. A model 
will be fitted that accounts for the greatest amount of variation in the 
judges' evaluations. If sufficient numbers of cases are available, inter- 
active effects and quadratic forms of the independent variables could be in- 
cluded in the initial set of predictor variables. The initial set of 
variables would be reduced using a stepwise or other appropriate procedure 
(Draper and Smith, 1966). An equation will be fitted for each year point 
at each level of experience. The coefficients of the fitted equations will 
provide a vector of weights for determining the two skill deterioration curves 
for any occupational classification or rating for which values of the pertin- 
ent skill deterioration factors can be determined. Thus, the resulting 
models are completely generalizable. 

Step 7. In order to evaluate the model and its weights, several un- 
related NECs should be used for a trial application. Again, experienced, 
senior personnel in the NECs will serve as judges. They will assign values 
to the skill deterioration factors in a step that is comparable to the oper- 
ation in constructing the hypothetical jobs (Step 4). Then, they will 
estimate the 5-year skill deterioration curves for the two experience levels, 
as was done in Step 5. Perhaps two or more methods should be used to aggre- 
gate the judgments. One would be a simple average of initial judgments, and 
another could be a consensus agreement arrived at, perhaps, using a Delphi 
technique. This recommendation is made since it may not be obvious as to 
what the best form of aggregating the responses might be in this exercise 
(Einhorn, Hogarth, & Klempner, 1977). Whatever the methods used, the 
model weights would be applied to the aggregated skill deterioration factors 
to predict the aggregated skill deterioration curves. The model will be 
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validated to the extent that the predicted values fall no further from the 
given values than would be expected from the model's error estimate at a 
specific confidence level, say at the 95 percent level. If the values 
fall outside of the confidence band, the validation procedures and the orig- 
inal model development will have to be reexamined to adjust the weight 
vectors . 

Application of the Model 

Once the weights have been validated, implementation of the model 
requires a set of occupational fields within the Navy's enlisted structure 
that is considerably less than the total number of ratings or NECs. Each 
occupational field within the set should encompass a group of occupations that 
is homogeneous so that the skill deterioration curves for any one A the 
occupations should be highly likely to reflect those of the other occupations 
within the category. Once the occupational fields are selected, an m x n_ 
matrix should be constructed in which the i_ rows (i_ = 1 to m) are the 
occupational fields, and the ^columns (j_ = 1 to nj are the skill deteriora- 
tion factors of the model. The problem, now, is to determine the scale 
values for each j_ x cell of the matrix. This could be done following the 
procedures that were used in the model validation (Step 7). The occupation 
that is most representative of the field, either by reason of being the lar- 
gest in number or qualitatively most encompassing of the other occupations 
or a combination of both of these considerations, should be used to obtain 
the scale values. In fact, the NECs that were used in the model validation 
could be transferred directly to the matrix within the appropriate occupa- 
tional fields. Once the matrix has been completed, the skill deterioration 
curves for any rating or NEC in the Navy can be computed for the two levels 
of experience--apprentice and journeyman/supervisor. 
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An important consideration in the implementation is the choice of a 
taxonomy for the rows of the matrix. The research literature emphasized 
task taxonomies, which are at too microscopic a level for this application. 

The Dictionary of Occupational Titles prepared by the Department of Labor 
would be more appropriate, since it is a classification of occupations. 
However, the classification is based on the universe of civilian occupations 
in the United States and may not be appropriate for the usage envisaged 
here. Probably the most appropriate classification would be the 23 occupa- 
tional fields devised by the NEOCS Study Group (1974). Each of the 23 
fields represents a functionally related set of ratings that could eventu- 
ally be converted to one rating in the apprentice stage. The purpose and 
philosophy behind the classification fits well with the purposes of the skill 
deterioration effort. A similar, but slightly more compact taxonomy, is the 
17-area, occupational specialty (OccSpec) system used by the Navy Recruiting 
Command for enlisting firs't term accessions into an occupational area without 
promising them a specific Class A School (Commander, Navy Recruiting Command, 
1974). The groupings of occupations in this system is dictated by the need 
to facilitate an even flow of recruits into the Class A Schools (Arima, 1976). 
As a consequence of this objective, the classification may not be as broadly 
conceived or appropriate as the NEOCS Study Group classification. 

Discussion 

This short-term approach is an interim framework for an eventual 
empirically based system. As stated in the discussion of the long-range 
program, the interim model cannot be validated or improved without a quan- 
titative measure of job performance that will enable a measure of skill 
loss. Similarly, the skill deterioration factors cannot be improved 
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without data so organized that task and relative importance data can be 
analyzed over all occupations simultaneously. While the long-range goals 
are sought, the interim method can be used as the best extant tool with 
the resulting experience in its application providing feedback for improv- 
ing it. 

Nothing has been said so far in this section regarding the real de- 
cision problem, which is the timing and quantity of skill maintenance 
training or retraining to be provided. As shown in Figure 1, it is desir- 
able to have a function that would predict the amount of retraining neces- 
sary to achieve the baseline performance from any point on the skill 
deterioration curve. Initially, this function could be derived using expert 
judges to capture their retraining policy. Eventually, the obtained func- 
tions would have to be supplanted with real-world data. To do this, the 
same impasse is reached: a quantitative measure of performance level is 

required and the variables to predict the required training should be based 
on an analysis of the actual characteristics of the training environment, 
whether it be on-the-job, self-study, or formal schooling. In the mean- 
while, retraining decisions could be made on the basis of the ski 1 1 -deteri - 
oration curves generated by the interim method and followed up to obtain 
feedback regarding the appropriateness of the decisions. If a cybernetic 
cycle can be established, the method will be self-correcting and will tend 
to reach an equilibrium (optimal) level, especially with the improvement 
in the data base that is envisaged. 
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SUMMARY AND RECOMMENDATIONS 



Skill deterioration among Navy personnel due to the nonutilization 
of learned skills is a serious problem with no easy solution. The avail- 
able scientific literature does not provide much that can be used to solve 
the problem. Empirical approaches to determining skill loss for the entire 
range of Navy occupations would be an undertaking of considerable propor- 
tions that would have to be done on a continuing basis as jobs change. 

Without the long-range commitment of resources and continuing command 
support, these methods will not produce the desired results. A short- 
range approach that attacks the problem directly using expert opinion 
seems to be the only feasible solution at this time. Along with the imple- 
mentation of the short-range approach, the Navy's performance measurement 
system (PQS) should be improved to permit a scoring of individual perform- 
ance that reflects the actual capabilities of the individual. The NOTAP 
program should be improved so that tasks are consistently described across 
all occupations to permit the analysis and derivation of important, com- 
posite skills using appropriate statistical procedures. Eventually, then, 
the short-range approach using personal judgments can be replaced with 
actual data in similar categories. In the meanwhile, training and re- 
training decisions should be made on the basis of the interim method and 
followed up so that a feedback cycle is established to ensure self-correction 
of decisions. 
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APPENDIX A 



AGENCIES CURRENTLY ENGAGED IN RESEARCH ON VARIOUS ASPECTS 
OF SKILL DETERIORATION AND RETRAINING 



Chief of Naval Education and Training (CNET ) 

NAS Pensacola, Florida 

Dr. Scanland, ACOS for Research and Program Development 
(AUTOVON 922-3466) 

Mr. Douglas Davis, Occupational Standards Training Programs 
(AUTOVON 922-3466) 

Mr. Hooprich, NAVPERSRANDCEN Liaison (AUTOVON 922-2621) 

Although CNET has no current program efforts devoted to 
research pertaining specifically to this study, there were 
helpful contacts made which merit attention for possible future 
reference. First, Dr. Scanland stressed the need for more in 
the area of assessing job performance which is a little under- 
stood art. From a training point of view, learning criteria 
reference measures should be patterned after suitable job 
performance measures (JPM) . JPM's are also good for helping 
to set standards and for instructional feedback. To accomplish 
this requires a synergistic training/performance system un- 
like the current diffuse subsystems that have no lines of 
communication between them. 

Mr. Davis is the JPM expert at CNET. He states that 
training curricula are developed from computerized job data 
gathered on Job Data Worksheets (Figure 1) . These worksheets 
are essentially a method of task analysis. There are 640 
task action codes that identify specific Navyvide skills. 

Mr. Hooprich is a liaison between CNET and several other 
Navy agencies. He is, thus, able to keep in close contact 
with all CNET projects and most training related projects 
throughout the Navy. He helped tremendously with this thesis 
effort and would be a good point of contact for any future 



efforts . 
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Figure 1. Job Data Worksheet 
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CNET Technical Analysis and Evaluation Group (TAEG ) 

NTC Orlando, Florida 

Dr. Smode, Director (AUTOVON 791-5198) 

TAEG is not currently engaged in any projects directly 
related to this study. However, Dr. Smode had several ideas 
worth noting on the subject of retraining. A possible approach 
to the problem should include the following three steps: 

1) Define the job requirements. 

2) Assess how well the worker can perform at the present 
time regardless of his nonutilization period variables. 

3) Retrain the individual to the required level. Histori- 
cally, collecting assessment, training, or skill deterioration 
data in an experimental setting has translated poorly to the 
operational environment. TAEG has had much success collecting 
actual operational data and believes this to be a much more 
viable route of progression. 
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Director of Naval Education and Training (DNET ) 

Arlington, Virginia 

Captain M. K. Mailhorne, Deputy Director, Programs 
Division (AUTOVON 222-4835) 

Captain Mailhorne is sponsoring the NPRDC study on the 
Performance Proficiency Assessment System, mentioned earlier. 

In addition to his position at DNET, he is also the Director 
for Plans and Policy for the Naval Reserve Readiness Command 
headquartered in Baltimore, Maryland. Though the NPRDC study 
on proficiency is longitudinal in nature. Captain Mailhorne 
believes a shorter term answer to skill deterioration is 
necessary. Some "gross cut" measure which will provide 
decision-makers a basis for determining retraining require- 
ments for large numbers of skilled personnel, especially within 
the reserve structure. All too often, the lengthy longitudinal 
studies lose momentum when key individuals move on to other 
assignments . 

In conjunction with his position in the reserve community, 
Capt. Mailhorne has submitted a point paper on training require- 
ments to maintain proficiency in the Reserves. The adequacy 
of the present drill structure and active duty period is in 
question and a study is called for to: 

" determine the effects of various drilling and 

ACDUTRA combinations upon the proficiency of post active 
duty drillers in various skill categories/field/applica- 
tions... The intent of the effort would be to obtain 
'ball park' information on the basis of which to exercise 
profesional judgement, rather than to create precise 
measurements and data." 

In essence, there is some doubt whether current reserve 
drilling maintains even the simplest of general military 
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skill, much less the technically oriented skill areas. 
Captain Mailhorne's views are shared by many senior officers 
in the reserve community, and the consequences of neglecting 
the expertise these views express could seriously degrade 
reserve readiness. 
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Fleet Anti-Submarine Warfare Training Center, Pacific 

San Diego, California 

Paul Asa-Dorian, Director of Curriculum Development and 

Standards (AUTOVON 225-4416) 

Mr. Asa-Dorian has been active in the training area for 
over twenty years, and has often cooperated with the Naval 
Personnel Research and Development Center (NPRDC) on joint 
effort studies. An example is skill retention of weapons 
systems operators, especially sonar and various other acoustic 
sensor operators. He has developed a modular training system 
for surface sonar operators utilizing self-paced instruction. 
This method is extremely effective in initial training of 
first tour sonar technicians (ST) , but has proven less than 
adequate for those personnel returning to the fleet for 
second or subsequent tours. Like many other ratings, ST's 
have little chance of maintaining their operational skills 
while on shore duty, unless they are assigned as instructors 
at an ASW training center. The ensuing skill loss from disuse 
is difficult to measure a posteriori and nearly impossible to 
predict beforehand. 

Mr. Asa-Dorian's problem is where to place personnel 
within the modular system upon their return from billets out 
of the sonar skill area. He believes some sort of performance 
measurement could prove beneficial in supplying data for the 
amount of retraining required within the modular system. 
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Naval Occupational Development and Analysis Center (NODAC) 

Washington, D.C. 

Mike Callahan, Navy Occupational Development Task Analysis 

Program (NOTAP) Department Head (AUTOVON 288-4621) 

C. T. Marshall, NOTAP Analyst (AUTOVON 288-4621) 

NOTAP was developed out of a requirement for task infor- 
mation data across all Navy enlisted billets. Collection of 
data commenced in 1967, and as of this year, approximately 
ninety percent of the enlisted rates and two officer categories 
have been processed and put into NOTAP data banks. The basic 
intent of this accumulation of job data is to identify what 
people do, not how well they do it. The development for each 
rating proceeds in the following manner: 

1) Subject matter experts (SME) from NODAC familiarize 
themselves with background information such as NEC manuals, 
Occupational Standards, rate manuals and PQS . 

2) The SME team then travels to a specified job site where 
an extensive use of the particular rating exists to obtain 
direct observations. 

3) A task inventory suvery is then developed and adminis- 
tered to 18-23% of the rating population. 

4) The results are reviewed, then placed on tapes and 
filed in a computer data bank. 

The types of data collected consist of such things as job 
satisfaction; primary and collateral duties; job characteristics; 
and most importantly, task statements which may be as numerous 
as 600. Access to the NOTAP data is handled by over forty 
subprograms which can analyze the data in various ways . 
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Correlation, cluster analysis, and matrix reduction are but 
a few of the options available. In essence, NOTAP offers an 
initial start in developing task analysis for a particular 
rate, and additionally some rather useful data analysis tools. 

Unfortunately, task criticality is not addressed as in 
the Air Force CODAP system. Also, user access must proceed 
via the Bureau of Naval Personnel, thus making any local 
real time response difficult, if not infeasible. However, 
the possibility exists of obtaining data tape copies on a 
specific rating for use at a local level. This would facili- 
tate any in-depth analysis at any Navy facility having computer 
capability . 
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Naval Personnel Research and Development Center (NPRDC) 



San Diego, California 

Adolph Anderson, Code 302 (714/225-2371, AUTOVON 933-2371) 

Mr. Anderson and his colleagues are in the process of 
developing a Performance Proficiency Assessment System for 
the Director of Naval Education and Training (DNET) . The 
system will have, as its primary objective, the determination 
of personnel readiness throughout the Navy. To this end the 
system's foundation has the following structure: 

1) Development of Job Performance Measures (JPM) : Two 
prototype rates, the Internal Communication Electrician (IC) 
and the Surface Sonar Technician (ST) , have been selected in 
conjunction with developing hands-on JPM's. These instru- 
ments will determine if the individual can accomplish the 
critical tasks within his job. 

2) Use of the quality control concept and sampling tech- 
niques: Since it would be expensive to administer JPM's to 

the entire rating population, sampling techniques will be 
employed to determine the degree of readiness within any 
particular rating. The aggregate of all ratings would thus 
quantitively indicate personnel readiness fleetwide. 

3) The prototype phase of the program will be completed 
in FY 1981, and at that time, other ratings will hopefully be 
considered for development. 

It is of interest to note that great emphasis was placed 
on task analysis in initial development, and extensive use 
of the Navy Occupational Task Analysis Program (NOTAP) preceded 
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identification of critical tasks. The Personnel Qualification 
Standard (PQS) material was also used, as was data on Navy 
Occupational Standards. The relevancy of this assessment 
system to this study lies in the mutual objective of deter- 
mining proficiency, or said another way, how well an individual 
retains his skill. The target populations differ (active 
duty in skill area versus various components out of skill 
area) , but the concept of a quantitative performance measure 
is quite amenable to ascertaining the degree of skill loss in 
populations both in and out of previously learned skill areas. 

In discussing skill retention in a more general sense, 
Anderson emphasized the following: 

1) The more rapid a person acquires a skill, the longer 
his retention. 

2) Continuous skills (tracking) are retained longer than 
discrete (procedural) skills. 

3) Obsolescence should be considered separate from skill 
deterioration. Skill deterioration usually takes place early 
on in the period of skill disuse while obsolescence is of 
longer term. 

4) The longer the retention interval, the greater the 
skill loss. This especially applies to procedurally oriented 
skills . 

In summary NPRDC and especially Anderson's group are study- 
ing areas of great mutual interest. Several contacts in other 
agencies were illicited, and continued exchange of information 
concerning job performance measures and assessment systems 
should prove to be quite beneficial. A good synopsis of the 



50 . 



NPRDC Performance Proficiency Assessment System was published 
by Pickering and Anderson [1976]. 
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Naval Reserve Readiness Command, Region Nine 
Memphis Tennessee 

Captain D. A. Nystrom, Commanding Officer (AUTOVON 966-5550) 

Because a large percentage of this study's target popu- 
lation is the reserve force, contact was made with a senior 
officer experienced in maintaining reserve readiness. The 
Selected Reserve of 104,000 is organized into sixteen Readi- 
ness Commands. These commands are structured under both 
functional and geographic guidelines. Captain Nystrom' s 
command (CTF-9) of approximately 6000 men is basically surface 
ship oriented and stretches from Kentucky down through Florida. 

Since the initiation of the total force concept in 1973, 
much effort has been made to increase reserve readiness and 
enhance the ability of the Selected Reserve to augment the 
active duty force. However, several deficiencies still exist 
in the area of training, specifically skills maintenance. The 
48 drill periods and 2 weeks active duty per year are proving 
to be adequate in the area of general military training, but 
fell short of insuring a sufficient amount of skill proficiency. 
Some of the more serious shortcomings are: 

1) No consistent funding for long range training programs. 

2) Geographical location of training centers within any 
Readiness Command prevent 100% participation in specific skill 
maintenance. 

3) Most equipment is obsolete. For example, communication 
components used in some training centers is of Korean War 
vintage . 
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4) Very little software or simulator systems to train 
weapons systems operators. In most cases, if personnel from 
the surface Selected Reserve were called up, they would have 
little or no knowledge of modern fleet systems. 

Captain Nystrom discussed a new program, the Versatile 
Training System (VTS) , which was initiated in 1972. Oriented 
towards enhancing enlisted aircraft maintenance training, VTS 
utilizes software simulation and self-paced instruction along 
with tailoring each individual's training by comprehensive 
performance tests (i.e., diagnostic, pre-tests, post-tests, 

PQS , etc.). Unfortunately, funding for remote terminal sites 
throughout Readiness Command Nine (pilot study) is unavailable 
at this time. Additionally, there is some disagreement as 
to VTS viability in the surface reserve, where many paygrades 
and rates are intermixed within any particular training center. 
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Naval Technical Training Center (NTTC ) 

Memphis, Tennessee 

Bob Coolidge, Program Manager for Other Service Veteran 

(OSVETS ) Retraining Program (AUTOVON 966-5955) 

The OSVETS program is designed to indoctrinate newly en- 
listed veterans of other services, in lieu of basic recruit 
training, in the procedures, policies, regulations, and 
traditions of the United States Navy.. The typical profile 
of the OSVET applicant is an E-4 with two years prior service, 
a "clean" record, and no deficiencies on the Basic Test Battery. 
The course length is three weeks and deals only with general 
military training such as firefighting, damage control, ship- 
board safety, and human resources management. The two OSVETS 
training sites, San Diego and Great Lakes, have a combined 
average annual input of 3500 to 4000 people. 

The intent of OSVETS, that of recapturing personnel re- 
sources, is quite sound. Although no specific skills are 
retrained, efforts are made to place OSVETS graduates into 
appropriate technical schools, particularly those located 
within the NTTC complex at Memphis. 

However, there are no quantitative methods to aid in 
screening OSVET applicants. In lieu of this, if the period 
of non or broken service exceeds two years, the individual is 
automatically sent back to recruit training. In addition, no 
performance measure is used to determine to what degree the 
OSVET graduate is qualified for technical training or 
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retraining. Here again skill retention data and methods to 
measure performance would greatly enhance the quality of the 
OSVET program. 
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Seville Research Corporation 



400 Plaza, Pensacola, Florida 

Dr. Wallace W. Prophet, Director (904/434-5241) 

Dr. Prophet was contacted because of his past work on 
retention of flying skills. Under the auspices of the Studies 
and Analysis Branch of USAF Headquarters, he authored an ex- 
tensive literature review and annotated bibliography mentioned 
in the preceding section [Prophet, 1976]. Although his present 
research group is not working in this area, he had several 
relevant comments. First, any retraining program must take 
into account the problem of individual differences. There 
are many variables comprising each person's nonutilization 
period which can interact in various ways to produce numerous 
retraining needs. He felt skill decay may be significantly 
checked through the use of low fidelity rehearsal devices. 

This type of program would especially be amenable to practicing 
procedural skills which are the most rapidly lost. In order 
for this idea to be successful, it was stressed that the 
Navy must provide sufficient incentive to motivate each in- 
dividual to keep current in his rate. Finally, Dr. Prophet 
agreed that performance assessment would be a useful approach 
to determine how much retraining was necessary provided a 
military agency developed the program to ensure its usefulness 
related to the operational environment. 



56 



U. S. Air Force Human Resources Laboratory 
Brooks AFB , San Antonio, Texas 

Dr. Raymond E. Christal, Occupation and Manpower Research 

Division (512/536-3648, AUTOVON 240-3648) 

For many years Dr. Christal has been a leader in research 
concerning enlisted skill acquisition, classification, and 
performance. One of his major achievements was creation of 
the Comprehensive Occupational Data Analysis Programs (CODAP) 
which is a computerized system for organizing and analyzing 
Air Force occupational information. Among other things, the 
system is capable of storing component tasks for each tech- 
nical rate along with the relative task criticality. Critical 
tasks are identified by examining three factors; a) consequences 
of bad performance, b) task delay tolerance, and c) task 
learning difficulty. Difficulty is defined as time to learn 
and is thought to be related to deterioration of the task. 

The primary data base is a task inventory conducted every six 
years. At this time Air Force jobs appear to be quite stable 
regarding the types of skills they require. Because of the 
CODAP system's proven usefulness, other services have adopted 
similar programs and the Navy system, NOTAP, will be discussed 
subsequently . 

Although Dr. Christal is not currently working on any 
project directly related to this study, his past experience 
and future plans are worth mentioning. His main interest is 
exploring the usefulness of aptitude tests which he recently 
discussed in a paper presented to the Military Testing Asso- 
ciation [Christal, 1976]. The main hypothesis is that 



57 



aptitude is a measure of the speed of skill acquisition. A 
corollary is that aptitude is also, then, a measure of re- 
learning time. Although it is felt that skill deterioration 
may be unrelated to aptitude directly, there are indirect 
connections between skill acqusition and skill decay. In this 
regard. Dr. Chris tal is beginning work on what he calls 
"perishability of skills." Past work supports the belief 
that those who learn faster and therefore attain greater 
proficiency, lose their skill level less rapidly. A future 
study will attempt to measure learning and decay and relate 
these to the Air Force aptitude testing battery. The question 
to be answered is the following: Is aptitude a good differ- 

ential for learning and decay rates? 
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U. S. Air Force Human Resources Laboratory 
Williams AFB, Phoenix, Arizona 

Dr. Edward E. Eddowes, Senior Technical Assistant 

(602/988-2611, AUTOVON 474-6604) 

Dr. Eddowes has devoted much of his energy toward pro- 
ficiency of flying skills. After the basic psychomotor skills 
involving flying are acquired, the concentration of training 
is on procedural type skills, not unlike those associated 
with Navy technical enlisted rates. It is agreed by most 
authorities that procedural skills deteriorate much more 
rapidly than psychomotor skills. Dr. Eddowes has recently 
completed a proposal for research on maintenance of flying 
skills. His attack focuses of reacquiring lost skill and on 
the amount of training sufficient to do the job. 

On the subject of performance assessment. Dr. Eddowes 
believes that the learning experience depends on viable per- 
formance evaluation to be successful. Because the system 
operator gets feedback from the system, the system output 
approach to assessment is often preferable to the control 
input approach. In other words the former approach measures 
final system behavior while the latter looks at what the 
operator does to the system. In essence an assessment program 
must first identify and define the skills involved, and then 
develop a means of measurement of those skills. This sequence 
of events will insure the relevancy of the performance measure 
to the task. 
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U. S. Army Research Institute (ARI ) 



Alexandria, Virginia 

Dr. Joyce Shields (AUTOVON 284-8695) 

Dr. Shields is working on a skill retention study involving 
Chaparral missile technicians. The Army has found this par- 
ticular MOS is lacking in demonstrated job proficiency in an 
operational environment. In order to answer questions in 
this area, a study has been designed to accomplish the following: 

1) Evaluate skill loss between training and utilization 
on the job. 

2) Determine the most effective refresher training. 

3) Provide data on forgetting over time for task performance. 

The study design is longitudinal in nature, and utilizes 

comparisons between control and noncontrol groups over varying 
retention intervals of zero, one, two, and four months. The 
basic intent is to gain data on the effects of nonpractice of 
a previously learned skill, which closely parallels the present 
effort of this paper. The maximum retention interval of four 
months is somewhat less than what is desired here, but other- 
wise the similarity is quite significant. 
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U. S. Army Training Support Center 



Fort Eustis, Virginia 

Capt Harry Porthouse, Individual Training Evaluation 

Group (AUTOVON 927-3128) 

The Training Support Center oversees and administers the 
Army's new Skill Qualification Test (SQT) program. The SQT 
replaces the former norm- referenced MOS test with a criteria- 
referenced performance battery. The program is essentially 
a quality assurance device designed to maximize combat 
effectiveness. It is also used to directly support training 
and the Enlisted Personnel Management System. SQT is based 
on the critical tasks identified in the Soldier's Manual and 
is administered in as many as three parts; written, hands-on, 
and performance certification. The program is very well 
organized and documented as demonstrated by the following 
sources: Ford, Campbell, and Harris [1976]; Maier, Young, 

and Hirshfeld [1976]; and Taylor and Vineburg [1975]. The 
Army seems to have the right idea by developing a compre- 
hensive program capable of use for performance evaluation, 
readiness assessment, personnel management, training, and 
advancement . 
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ADDENDUM: FORMAL MODEL 



ASSUMPTIONS 

1. For all occupations, there is a limited number of independent 
factors that determine the skill deterioration rate with the nonutilization 
of ski 11s. 

2. These skill deterioration factors have weights that determine their 
relative effects independent of any occupation. 

3. The weights will differ according to the initial skill level of 
the person and the period of nonutilization of skills. 

4. All occupations can be scaled as to the degree of presence or 
relative importance of each skill deterioration factor. 

MODEL 

The skill deterioration that can be expected for a set of occupations 
after a period of nonutilization and an initial skill level can be ex- 
pressed as: a r = r 



where A is a matrix of occupations by values for their skill deterioration 
factors, B is a matrix of weights for the skill deterioration factors by 
a particular initial skill level and period of nonutilization, and C is a 
matrix of expected skill deterioration for each occupation by initial skill 
level and period of nonutilization. Figures A-la through A-lc show a 
schematic representation of the model. 



i x j j x k 



C 

i x k 



( 1 ) 
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j 

Skill Deterioration Factors 




Figure A-la. Matrix (A) of Values for the Skill Deterioration Factors 
by Occupations 



ESTIMATION 

The problem is to determine the skill deterioration weights. This can 
be accomplished by creating synthetic (hypothetical) occupations portraying 
the values in the A(i x j) matrix and having a number (n) of experts 
estimate the skill deterioration of those occupations in the C(i x k) 
matrix. The weights for the B(j x k) matrix will then be obtained by 
linear regression of the following form: 



C ik = a o + b jk X f j + u ik 

where 



( 2 ) 
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Level of Initial Skill and Period of Nonutilization 
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tice and journeyman; the periods, years. 



Level of Initial Skill and Periods of Nonutilization 



1 
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of Initial Skill and Period of Nonutilization. 



i = 1 , 2, - - - i synthetic occupations 

j = 1, 2, - - - j skill deterioration factors 

k = 1 , 2 skill levels and 1 , 2, - - - k years of nonutilization 
nested within each level of initial skill 

A 

C = estimate of the judges' skill deterioration judgments 

a = constant to be estimated from the data of judgments 

b = the j x k skill deterioration weights to be estimated from the data 
X = synthetic value for occupation i of skill deterioration factor j 
u = residual error for occupation i at skill level 1 or 2 and non- 
utilization period k due to the unsystematic variation in judg- 
ments of C by the n judges 

The best- fitting equation can be determined by an examination of the resid- 
uals and the standard error of estimate of the prediction equation. 

APPLICATION 

Once the skill deterioration weights of the B matrix have been deter- 
mined, expert judges can be used to scale the presence or relative importance 
of the skill deterioration factors for real occupations or occupational 
groups, thus completing matrix A . Matrix C, skill deterioration, is then 
given by the model. 

The model provides the framework for updating and refining the judgment- 
ally derived functions with empirically derived data as it becomes available 
through improved and integrated NOTAP and PQS systems. 
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